Repository navigation
FEAT: Private targets that only their creator can see and use - #4024
Conversation
A registered OpenAI-family (except RealtimeTarget) or LiteLLMChatTarget created
through POST /api/targets with an api_key no longer keeps that key. The backend
stores it in process memory for the creating caller and registers the target
with a provider that returns the current caller's own credential on every
request. Other callers can see and use the target but must set their own
credential; without one their requests fail closed with
CallerCredentialRequiredError instead of borrowing another caller's key or
falling back to server credentials.
- pyrit.auth.caller_credential: CallerCredentialStore, scoped caller principal,
and fail-closed provider.
- Auth middleware scopes each request (and work it starts) to its caller.
- New routes: PUT/DELETE /api/targets/{id}/credentials, DELETE /api/targets/{id}.
- Queued scenario runs keep their admitting caller; shared estimates run with none.
- Credentials are cleared on service closure and runtime reset.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Extra body parameters can override the resolved caller API key, causing environment-based credential fallback or use of a shared static key instead of the caller’s credential.
1 open issue
What changed in this PR
This PR introduces per-caller credentials for shared CoPyRIT targets, separating submitted keys from target configuration. It changes Entra deployments so users must configure their own credentials for supported targets.
Changes:
- Adds an in-memory credential store and caller-scoped providers for OpenAI-family targets, excluding Realtime, and LiteLLM.
- Adds credential-management routes, creator-only target deletion, and caller-specific access fields.
- Propagates caller identity into queued work and adds lifecycle cleanup, regression tests, and documentation.
| File | Description |
|---|---|
tests/unit/prompt_target/target/test_litellm_chat_target.py |
Tests caller-specific LiteLLM credentials. |
tests/unit/prompt_target/target/test_caller_credential_targets.py |
Tests OpenAI concurrency, replacement, and fail-closed behavior. |
tests/unit/backend/test_temporary_components.py |
Updates the registry factory mock signature. |
tests/unit/backend/test_target_service.py |
Updates registry delegation expectations. |
tests/unit/backend/test_target_caller_credentials.py |
Tests credential lifecycle and REST access controls. |
tests/unit/backend/test_scenario_service.py |
Tests caller-free shared estimates. |
tests/unit/backend/test_scenario_run_service.py |
Tests caller identity across queued runs. |
tests/unit/backend/test_auth_middleware.py |
Tests principal derivation and task propagation. |
tests/unit/auth/test_caller_credential.py |
Tests credential storage, scopes, and providers. |
pyrit/registry/registry.py |
Allows trusted server-side constructor parameters. |
pyrit/prompt_target/openai/openai_target.py |
Enables caller credentials for OpenAI targets. |
pyrit/prompt_target/openai/openai_realtime_target.py |
Excludes persistent Realtime connections. |
pyrit/prompt_target/litellm_chat_target.py |
Enables caller credentials for LiteLLM. |
pyrit/prompt_target/common/prompt_target.py |
Defines the target credential opt-in contract. |
pyrit/models/catalog/target.py |
Adds caller-specific credential and deletion fields. |
pyrit/backend/services/target_service.py |
Manages credential injection, ownership, and deletion. |
pyrit/backend/services/service_lifecycle.py |
Clears credentials during service closure. |
pyrit/backend/services/scenario_service.py |
Removes caller identity from shared estimates. |
pyrit/backend/services/scenario_run_service.py |
Preserves admitting callers for queued execution. |
pyrit/backend/routes/targets.py |
Adds credential and target deletion endpoints. |
pyrit/backend/README.md |
Documents credential behavior and API usage. |
pyrit/backend/models/targets.py |
Adds the write-only credential request model. |
pyrit/backend/models/__init__.py |
Exports the credential request model. |
pyrit/backend/middleware/auth.py |
Scopes requests to stable caller principals. |
pyrit/auth/caller_credential.py |
Implements credential storage and fail-closed providers. |
pyrit/auth/__init__.py |
Exports caller-credential helpers. |
doc/code/framework.md |
Clarifies auth and backend responsibilities. |
Review effort: Balanced
Richard Lundeen (richlundeen)
left a comment
There was a problem hiding this comment.
Automated code_review at 8c7c8fc43579425456623feec9bfb02ef355aec3: two P1 and two P2 findings below. The existing LiteLLM passthrough concern is retained without a duplicate comment.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Targets stay shared by default. A target uses per-caller credentials only when created with requires_caller_credential=true, for providers that need a personal token. Unsupported target types, auth modes, and unregistered targets are rejected when that option is requested; the creator may omit the key and set it later. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Fixed in Addressed this review. |
|
Fixed all four findings in Addressed this review. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Richard Lundeen (richlundeen)
left a comment
There was a problem hiding this comment.
Automated review completed. No blocking concerns remain. This review records source coverage; it does not authorize merge.
- pyrit_scan set-target-credential / remove-target-credential / add-personal-target read a personal token from a hidden prompt, an environment variable, or the Anthropic ant CLI (ant auth print-credentials --access-token, configurable --ant-profile) and send it to the backend. --keep-fresh re-sends a fresh ant token on an interval until Ctrl+C. Tokens are never command-line values, never printed, and never sent to a remote non-HTTPS backend. - POST /api/targets with source + register + requires_caller_credential registers a personal copy of an existing target, such as one an initializer configured. The copy keeps the source's settings but replaces its key, token provider, or managed identity with each caller's own credential. - Registry.recreate_instance accepts server_params; LiteLLMChatTarget is reconstructable. - list-targets shows whether a target is personal and whether your credential is set. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Richard Lundeen (richlundeen)
left a comment
There was a problem hiding this comment.
Automated review completed. No blocking concerns remain. This review records source coverage; it does not authorize merge.
Replace the Only me switch at the top of Create New Target with a "Who can use it" choice (Everyone or Only me) after the key field, so the form ends with who gets the target and both options are visible. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cover both send APIs, a target named in the chat and a saved binding, a signed-out request, and the owner still sending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks Behnam (@behnam-o). This PR has been redesigned since your review, which changes the picture. There's no
|
varunj-msft
left a comment
There was a problem hiding this comment.
Thanks Rich! I went through this and checked each finding myself. The access check looks solid: every lookup, listing and reference path I traced hides other people's private entries, and privacy spreads to composites as described. At this head the backend, registry, auth, cli and prompt_target suites pass, and so do the changed jest files.
I left 7 inline comments. The ones I'd fix before merging: the keep-fresh loop, the identity to key switch, components keeping the old token, LiteLLM losing detected capabilities, and private names blocking run-time initializers. The two doc ones are quick.
Part 8 is in main now (#4026, merged today), so this PR conflicts with it in 7 files. Things to watch in that merge:
- Part 8 adds
persisttoPOST /api/targets. Withprivate: true, the recipe has no private flag and restore runs with no caller, so the target comes back shared after a restart. Keys are still refused, so no token gets written, but the target itself becomes everyone's. I'd refuseprivatetogether withpersistfor now. - Keep
creator_metadataon part 8's persist branch, otherwise a saved target loses its owner right away. - The credentials PUT should probably also refuse configuration targets (
registry_entry_origin(...)isCONFIGURATION), since changes to those are lost on restart.
For closing #4028 (part 10): the creator record here covers targets, converters, scorers and techniques, which is what we need. To move our later parts onto it we'd still need:
- A way to show "you created this" for shared entries. Today the API only has
private, and only on targets, so my own shared target shows as Shared, and private scorers, converters and techniques aren't marked at all. - Restore keeping who saved an entry. The recipe already stores the saver's oid and name, so restore could register with
owner_metadata(...)without running as that user. It needs a helper that builds the principal from an oid (plus the local mode case). - A display name, if we want "Saved by Alice".
I'm happy to do these in our stack after this lands so this PR stays focused. Does that work for you?
One question: should updating a shared target's credentials be limited to its creator or an admin? Right now anyone signed in can change the key on a target from an initializer. Parts 20 and 21 use owner or admin for editing and deleting saved entries, so it would be nice to share one rule.
Bashir Partovi (bashirpartovi)
left a comment
There was a problem hiding this comment.
It looks like pasting a key with a trailing space dumps the whole key into shared conversation results.
If a key comes in through POST /api/targets or PUT /api/targets/{name}/credentials with a trailing space or newline (super common when pasting), we keep it exactly as the user typed. The first request through that target then fails on the HTTP layer, and the error text we save into the conversation contains the full key. Anyone who can read results can read the key. If you run the code below, it exactly reproduces this case.
Just to explain the flow,
- We only check that the value isn't blank:
pyrit/backend/services/target_service.py:814in_validate_credential_update, and the same idea at:651on create. The original string, trailing space and all, is what the target gets built with.pyrit/auth/openai_auth.py:80returns the exact key user pasted andpyrit/prompt_target/openai/openai_target.py:768passes it straight intoAsyncOpenAI. - On the first send the SDK builds
Authorization: Bearer <key>, and h11 refuses the header value withIllegal header value b'Bearer sk-... '. That message has the full key in it (h11/_headers.py:166). httpcore2 and httpx2 pass it through, the SDK wraps it asAPIConnectionErrorwith the cause attached (openai/_base_client.py:1840), and our handler just logs and re-raises (openai_target.py:884-888). PromptNormalizercatches it and savestraceback.format_exc()as an error message piece (pyrit/prompt_normalizer/prompt_normalizer.py:230-239). So the key is now sitting inPromptMemoryEntries.original_value. The attack result gets a second copy inerror_traceback(pyrit/executor/attack/core/attack_strategy.py:696).GET /api/attacks/{id}/messages?conversation_id=...hands that piece back as-is (pyrit/backend/routes/attacks.py:460,pyrit/backend/mappers/attack_mappers.py:331-335), and results are visible to everyone on purpose. Rotating or removing the key later doesn't touch those rows. And because the error has no status code,credentials_rejectedstays false, so the Targets table keeps showing a happy green key.
The CLI is actually fine here, since pyrit_scan set-target-credential strips the value and rejects internal whitespace (pyrit/cli/_target_credentials.py:127-131). The GUI create dialog sends the field untrimmed though (frontend/src/components/Config/CreateTargetDialog.tsx:664), and anyone hitting the API directly obviously can too.
To see it yourself (REST against a local backend)
# 1. Anything that accepts a TCP connection. h11 bails before a byte is sent, so the server doesn't matter.
python -m http.server 8099 &
# 2. Backend in local mode.
pyrit_scan start-server
# 3. Make a target whose key ends in a space. PUT /api/targets/<name>/credentials with the same value works too.
curl -s -X POST http://localhost:8000/api/targets -H 'Content-Type: application/json' -d '{
"name": "leaky-demo", "type": "OpenAIChatTarget",
"params": {"endpoint": "http://127.0.0.1:8099/v1", "model_name": "gpt-test",
"api_key": "sk-synthetic-not-a-real-key-000000 "}}'
# 4. Send one message through it (or just chat with it in the GUI). Grab attack_result_id and conversation_id.
curl -s -X POST http://localhost:8000/api/attacks -H 'Content-Type: application/json' \
-d '{"target_registry_name": "leaky-demo"}'
curl -s -X POST http://localhost:8000/api/attacks/<attack_result_id>/messages -H 'Content-Type: application/json' \
-d '{"role": "user", "pieces": [{"original_value": "hello"}]}'
# 5. Read the conversation back as anyone.
curl -s "http://localhost:8000/api/attacks/<attack_result_id>/messages?conversation_id=<conversation_id>" \
| grep -o "Illegal header value[^\"]*"Step 5 gives you Illegal header value b'Bearer sk-synthetic-not-a-real-key-000000 '.
Standalone script, if you'd rather not spin up the backend.
Run it with RETRY_MAX_NUM_ATTEMPTS=1 RETRY_WAIT_MIN_SECONDS=0 RETRY_WAIT_MAX_SECONDS=0, otherwise the retry decorator sits there backing off for minutes.
"""Reproduce: an API key with a trailing space ends up persisted in an error message piece."""
import asyncio, socket, tempfile, threading, uuid
from pathlib import Path
# A listener that accepts and closes, so the request reaches header serialization (h11) instead of failing to connect.
srv = socket.socket(); srv.bind(("127.0.0.1", 0)); srv.listen(5); PORT = srv.getsockname()[1]
def serve():
while True:
try:
conn, _ = srv.accept(); conn.close()
except OSError:
return
threading.Thread(target=serve, daemon=True).start()
KEY = "sk-synthetic-not-a-real-key-000000 " # trailing space, as pasted from a clipboard
async def main() -> None:
from pyrit.memory import CentralMemory, SQLiteMemory
from pyrit.models import Message, MessagePiece
from pyrit.prompt_normalizer import PromptNormalizer
from pyrit.prompt_target import OpenAIChatTarget
db_path = Path(tempfile.mkdtemp(prefix="pyrit-repro-")) / "repro.db"
memory = SQLiteMemory(db_path=db_path, silent=True)
CentralMemory.set_memory_instance(memory)
# Same constructor call the backend makes for POST /api/targets or PUT /api/targets/{name}/credentials.
target = OpenAIChatTarget(endpoint=f"http://127.0.0.1:{PORT}/v1", model_name="gpt-test", api_key=KEY)
conversation_id = str(uuid.uuid4())
message = Message(message_pieces=[MessagePiece(role="user", original_value="hello", conversation_id=conversation_id)])
try:
await PromptNormalizer().send_prompt_async(message=message, target=target, conversation_id=conversation_id)
print("UNEXPECTED: send succeeded")
except Exception as ex:
print(f"send raised {type(ex).__name__}: {str(ex)[:70]}")
pieces = await memory.get_message_pieces_async(conversation_id=conver
print(f"message pieces persisted for the conversation: {len(pieces)}")
for piece in pieces:
if piece.original_value_data_type == "error":
print("error piece persisted; synthetic key present in originece.original_value)
for line in piece.original_value.splitlines():
if "Illegal header value" in line:
print(" persisted line:", line.strip()[:150])
break
print("credentials_rejected on the target:", target.credentials_rejected)
await memory.dispose_engine_async()
asyncio.run(main())What I got:
send raised Exception: Error sending prompt with conversation ID: cf6719aa-df9f-42c1-82a7-0c7
message pieces persisted for the conversation: 2
error piece persisted; synthetic key present in original_value: True
persisted line: httpcore2.LocalProtocolError: Illegal header value b'Bearer sk-synthetic-not-a-real-key-000000 '
credentials_rejected on the target: False
If you add a trailing newline or an embedded NUL, you can repro the same thing.
I think the way to fix this is:
- In
_validate_credential_updateand the create path, strip the value, then reject it if anything left is whitespace or non-printable. Keep the error to the parameter name
only. If we hang it off every parameter markedsensitive(`get_credentiists them), tokens and headers get the same treatment. Trimming in the GUIdialog wouldn't hurt either. - We can add a regression test that PUTs
"sk-x ","sk-x\n"and"sk-x\x00", expects a 400 each time, and checks the old key is still in place. Also one at the normalizer level asserting a saved error piece never contains the key. - I think we can also add some enhancements in how we handle sensitive header values, we could redact
Authorizationvalues in the text we build atprompt_normalizer.py:230. That sink was there before this PR, but this PR adds a second way in and a workflow built around pasting tokens a lot, so it feels like the right time to close it off.
Behnam (behnam-o)
left a comment
There was a problem hiding this comment.
The registry-level access check and privacy propagation hold up well. Every lookup, listing, and reference path I traced hides other callers' private entries.
This review has two parts, both as inline comments so each can be acknowledged and resolved:
- Redesign suggestions D1–D4. D1 and D2 are the main proposal: allow credential updates only on the caller's own private targets, and update credentials in place through a token provider instead of rebuilding the target. D3 suggests splitting rejected-credential reporting into its own PR. D4 is non-blocking.
- Findings F1–F10, numbered so they can be referenced.
If D1 and D2 are adopted, F1–F5 go away and need no separate fixes:
- D1 addresses F4 and F5.
- D2 addresses F1, F2 and F3.
F6–F10 apply either way. F6 and F10 move with D3 if rejected-credential reporting is split out.
Reviewed at 9b0223b8.
…targets Keep both behaviors in the conflicting files. A target response reports both `private` and part 8's `origin`. A saved target registered through `persist` keeps the creator metadata, so it records its creator and privacy. The registry keeps part 8's `registry_entry_origin` and configuration marker along with this PR's owner and private-entry checks. The PR's new private-registration checks use main's `register_instance` field. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A target records whether its credentials can be updated (allow_credential_updates): private
targets allow it by default for their owner, shared targets only when their creator or an
initializer (credential_update_metadata) allows it, and then for anyone signed in.
- PUT /api/targets/{name}/credentials takes api_key and any request headers (a cookie is the
Cookie header) and changes the registered target in place: the OpenAI client is swapped with
with_options, so everything holding the target uses the new values. The auth mode never
changes, framing headers are refused, and a "private" precondition returns 409 on mismatch.
- Keys, sensitive parameters, and header values are stripped and refused with control characters
on create and update, so a pasted newline can't leak through an HTTP error.
- credentials_rejected is null until a request completes and after an update; 401/403 from an
httpx.HTTPStatusError count as rejections.
- A launch script replacing a name a private entry uses reports a clear conflict; techniques of
different owners can share a tag; private or updatable targets can't be persisted.
- Remove rebuild_instance and the LiteLLM reconstruction that only the rebuild needed.
- CLI: set-target-credential --header, refuses targets that don't allow updates, and keep-fresh
stops when the target's visibility changes. GUI: an "Allow updates" switch, no credential icon
before first use, and the lock explains when a target can't be updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…update An initializer can build an Entra ID target by passing a token provider as api_key with the default auth_mode "api_key". An update refuses api_key and an Authorization header for any target whose key is a token provider, not only auth_mode "identity", so the update can't swap everyone's requests from the provider to a static key. Header values may contain a tab, as HTTP allows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
varunj-msft: main, including part 8, is merged in Bashir Partovi (@bashirpartovi): fixed.
Tests: Behnam (@behnam-o): each D and F item has a reply in its thread. Summary: D1 is adopted as a per-target Verified on published commit |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
varunj-msft
left a comment
There was a problem hiding this comment.
Thanks Rich! I re-checked everything at 43236455a. All 7 of my threads are fixed (replies in each), and the part 8 merge looks right. The saved target keeps its creator. private or allow_credential_updates together with persist is refused, and a round robin that's private because it uses a private target is still refused by part 8's Temporary rule. Our part 8 tests are unchanged. The backend, registry, auth, cli, prompt_target, and setup suites pass at this head (10,836 tests).
One thing I'd fix before merging: a key or header with a non-ASCII character still leaks into shared results through the UnicodeEncodeError the normalizer saves (inline on credential_values.py). It should be a small fix. The other four comments are minor.
Happy to approve once the leak is fixed.
Behnam (behnam-o)
left a comment
There was a problem hiding this comment.
LGTM, thank you!
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Thanks, Varun. Credential input now rejects non-ASCII values without echoing them. I also addressed the status, Authorization, and launch-script follow-ups and strengthened the held-client regression. Verified on published commit |

Description
Before this change, CoPyRIT users had no good way to use a target with their own token: a target's credentials were fixed when it was created, short-lived tokens (such as an Anthropic
antlogin) expired with no way to refresh them, and every target someone created could be used by everyone. This lets a user keep a target to themselves, and lets a client keep a target's credentials up to date where whoever created the target allowed it.allow_credential_updateswhen it's created. A private target allows updates by default if its type can make them in place (OpenAI-family targets other thanRealtimeTarget), and only its owner can make them. A shared target doesn't, unless whoever created it allows it; then anyone signed in can. An initializer can allow it for a target it registers withcredential_update_metadata(allowed=True), for example a target whose gateway token, in an extra header, expires. Saved targets can't be updated.PUT /api/targets/{id}/credentialstakes anapi_key, any requestheaders(a cookie is theCookieheader), or both. It replaces them on the registered target without rebuilding it, so its identity, configuration, detected capabilities, and auth mode stay the same, and everything using it sends the new values with its next request. A header replaces one of the same name, whatever its case, whether it was configured inheaders,OPENAI_ADDITIONAL_REQUEST_HEADERS, orhttpx_client_kwargs. A target that gets its tokens from a token provider, such as Entra ID, keeps it. Aprivatevalue in the request makes the update fail with 409 if the target isn't private or shared as expected.pyrit_scan set-target-credentialreads the new value from a hidden prompt, an environment variable, or theantCLI, sets it as the key or, with--header, as a header, and--keep-freshre-sends a freshanttoken until Ctrl+C.credentials_rejectedafter the provider answers 401 or 403, the GUI shows a red lock on it with how to update the credentials (or that the target doesn't allow updates), andlist-targetssays so. It'snull, with no icon, until a request completes, and again after a credential update."private": true(or Only me in the GUI) can be seen and used only by its creator. Targets stay shared by default, visibility never depends on the credentials a target uses, and a private target can't be saved.How it works
GUI changes
Registry > Targets has a new Account column: "Shared" or "Yours", with a green key when the provider accepted the target's credentials, a red lock when it rejected them, and no icon until the target is used. An Account filter sits beside the Type filter.
Clicking the red lock explains how to update the credentials, or, for a target that doesn't allow updates, to ask whoever set it up:
Create New Target asks Who can use it (Everyone, the default, or Only me) after the sign-in fields, then, for OpenAI-family targets other than
RealtimeTarget, whether to Allow updates to its credentials, which follows Only me unless you change it. A round robin that uses one of your private targets stays Only me, with a hint:🤖 Generated with Claude Code